Perception of Evaluators While Evaluating University Theory Answer Sheets: Cross-Sectional Virtual Survey

Neetu Bhatnagar¹, Bharti Sharma², Sain Priya³

¹Assistant Professor, Department of Mental Health Nursing, Bee Enn Nursing Institute, Jammu.

²Assistant Professor, Department of Medical-Surgical Nursing, Bee Enn College of Nursing, Jammu.

³Associate professor, Department of Child Health Nursing, Bee Enn College of Nursing, Jammu.

*Corresponding Author E-mail: bhartisharma7294@gmail.com, sainpriyaofficial123@gmail.com

ABSTRACT:

Background: Nursing is a profession in which student after completing their training programme and provides best care to the patients. This training will shape the student’s future and make it bright. This Profession makes a student’s more decision maker, critical thinker and deal with daily ethical challenges. An important and simultaneously difficult aspect of good examination preparation is obtaining a properly trained and well-motivated group of assessors. To effectively recruit and maintain cooperation with assessors, it is worth knowing their opinion. The main aim of conducting this survey is to know the opinion of evaluators while evaluating.  Objective: To assess the perception of evaluators while evaluating university theory answer sheets and to find out the association between the perception of evaluators with their selected demo-graphic variables. Method: A cross-sectional study was conducted using a questionnaire on evaluators who evaluate university theory answer sheets. This questionnaire consisted of 12 dichotomous questions about their perceptions. Purposive sampling technique was used to collect data. A total of 114 evaluators participated in this study. Descriptive and inferential statistics was used to find out the result. Finding: The Result showed that out of 114 evaluators, maximum 56.1% of evaluators were belongs to the age group of 31-40 years followed by 82.5% were females. Majority 93.9% were working in private area followed by 78.9% were qualified M.Sc (N). Majority 38.6% of evaluators were having 3 to7 years of experience. Majority of the evaluators has shown positive perception regarding all the aspects except the good or bad mood of the evaluator while checking the university theory answer sheets and checking previous theory sheet will affect the marking of the next answer sheets. Conclusion: The study concluded that the variables like age, gender, working area, qualification and years of experience were found to be statistically non-significant i.e., p<0.05.

KEYWORDS: Perception, Evaluator, Answer sheets, Evaluation.

INTRODUCTION:

Despite the abundant literature on educational measurement there has been relatively little work investigating the psychological processes underpinning marking. Person perception is a term utilized in social psychology to delineate the cognitive processes involved in shaping impressions of individuals. This encompasses not only the formation of these impressions but also the diverse inferences drawn from them. Reflect on the frequency with which you engage in such evaluations daily ¹.

Perception is the human process of organizing sensory input into coherent experiences. These experiences, known as precepts, result from both external stimuli and internal cognitive processes. By examining relationships between different stimuli and their corresponding precepts, researchers can develop theories about how perception operates. Since perception occurs privately within the mind, its theories are evaluated through indirect means, primarily through experimental research where theoretical predictions are compared with empirical data².

NEED OF THE STUDY:

The examination process is a critical component of any educational institution, involving the evaluation and grading of students. Traditional methods of assessing theory exams entail numerous manual tasks and pose challenges, particularly regarding safeguarding students' identities to prevent biases. This necessitates concealing identifying information like roll numbers or names, leading to increased logistical burdens. Examiners typically need to visit specific locations for evaluation, and manual compilation of results further prolongs the process, often resulting in delays in result announcements³.

Assessing answer sheets is among the most time-intensive responsibilities educators face. Throughout their professional journeys, teachers and evaluators invest a significant portion of their time in the digital assessment of answer sheets. Given the widespread adoption of digital technologies in schools and universities globally, online evaluation of answer sheets has become customary. This approach offers numerous benefits⁴.

PROBLEM STATEMENT:

A study to assess the perception of evaluators while evaluating university theory answers sheets.

OBJECTIVES:

1.   To assess the perception of evaluators while evaluating university theory answers sheets.

2.   To find out the association between the perceptions of evaluators while evaluating theory answer sheets with their selected socio-demographic variables.

ASSUMPTION:

It is assumed that the evaluators may have positive or negative perception while evaluating theory answer sheets.

DELIMITATIONS:

The study was limited to:

· Evaluators who are working in nursing field.

· evaluators who are evaluating sheets for degree courses in nursing.

REVIEW OF LITERATURE:

Henok Fisseha, Hailemichael Desalegn (2021) conducted a study on Perception of Students and Examiners about Objective Structured Clinical Examination in a Teaching Hospital in Ethiopia. The aim of this study was to assess the perception of students and examiners towards OSCE. A cross-sectional study was conducted on students and examiners undertaking OSCE from May 1 to July 30, 2021, using a structured questionnaire. A total of 141 students and 39 examiners participated in the study. The majority of the students and examiners had a positive response regarding the attributes, structure, organization and validity of OSCE. It was recommended to be used in future exams compared to other assessments by 38.3% of students and 51.3% of examiners. There were certain challenges reported by students and included stressfulness of the exam (51.1%), inadequate time (27.6%), and unsatisfactory orientation (30.5%). One-third of examiners considered it stressful, while 20.5% considered the time provided to be inadequate. Equipment to conduct the exam was considered inadequate by 39.1% and 56.4% of students and examiners, respectively. Around 80.1% of students recommended mock sessions and 23.1% of examiners did not have any prior training on OSCE. The study concluded that an overall positive perception of OSCE by students and examiners was seen. Certain challenges that need improvements were identified. Continuing evaluation and refinement of OSCE by departments is needed. We recommend further wide-scale national evaluation of the OSCE examination system of medical students⁵.

Agata Stalmach-Przygoda, Michał Nowakowski, Anna Kocurek, Ian Perera, Agnieszka Skrzypek, Jadwiga Mirecka, et.al (2020) conducted a study to assess the Perceptions of Clinical Teachers Acting As Examiners Regarding The Value of Objective Structured Clinical Examinations in Poland. The objectives of the study were to effectively recruit and maintain cooperation with assessors, it is worth knowing their opinion. The aim of this study was to investigate the opinions of teacher-examiners about the OSCE and to identify the factors that could shape this opinion and influence on motivation. A cross-sectional study was conducted using a questionnaire on teachers who participated as OSCE examiners. This questionnaire consisted of 21 questions about their perceptions. Answers were rated in a five-point Likert-type scale. Results revealed that a total of 49(out of 52) teachers participated in this study. Nearly 90% of examiners believed that it is fair, and more than 90% that it is transparent. Despite the fact that 67% of examiners believe that the examination is difficult to organize and 71% believe it is stressful for students; according to 72% of respondents the OSCE has a positive effect on learning. More than 91% of examiners believed that the OSCE is an appropriate test to assess students’ skills. Opinions about the examination were independent of specialty, seniority, gender or having taken the OSCE as students. Therefore, teacher-examiners viewed the OSCE as a fair and transparent examination, adequate for the assessment of skills and, despite it being difficult to organize, worth doing as it is appropriate to assess practical skills and positively influences students’ motivation to learn tested skills⁶.

Margery H. Davis and Gominda G. Poonamperuma: (2010) conducted a study on Examiner Perceptions of A Portfolio Assessment Process in Sri Lanka. The main aim was to study examiner perceptions of Dundee Medical School's portfolio assessment process, in years 4 and 5 of the 5-year curriculum, in relation to: outcomes as a framework for the portfolio assessment process; portfolio content; portfolio assessment process; end points of the portfolio assessment process; appropriateness of the two part final exam format and examiner training. A questionnaire containing statements and open questions was used to obtain examiner feedback. Responses to each statement were compared over 3 years: 1999, 2000 and 2003. Response rates were 100%, 88% and 61% in 1999, 2002 and 2003, respectively. Examiners were positive about the ability of institutionally set learning outcomes (Dundee 12 exit learning outcomes) to provide a framework for the portfolio assessment process. They found difficulties, however, with the volume of portfolio content and the time allocated to assess it. Agreeing a grade for each learning outcome for the candidate with their co-examiner did not present difficulties. The comprehensive, holistic picture of the candidate provided by the portfolio assessment process was perceived to be one of its strengths. Examiners were supportive of the final examination format, and were satisfied with their briefing about the process. The 12 exit learning outcomes of Dundee curriculum provide an appropriate framework for the portfolio assessment process, but the content of the portfolio requires fine-tuning particularly with regard to quantity. Time allocated to examiners for the portfolio assessment process needs to be balanced against practicability. The holistic picture of the candidate provided by the process was one of its strengths⁷.

Victoria Crisp (2008) conducted a study to Exploring The Nature of Examiner Thinking During The Process Of Examination Marking in Cambridge. This research investigated the processes involved when examiners mark examination responses. Scripts from two geography A‐level examinations were used: one requiring short and medium length responses and one requiring essays. Six examiners marked 50 scripts from each of the two examinations and were later asked to think aloud whilst marking four to six scripts from each examination. Coding and analyses identified different types of reading behaviours, social, emotional and personal reactions and provided insight into the nature of evaluations. Some differences between examiners and between question types were identified. Analysis of associations between marker behaviours and marker agreement suggested that positive evaluations, comparisons and thorough reading were important to avoiding severity. Potential implications for marker training and for the impact of technological changes to assessment systems are discussed⁸.

MATERIAL AND METHODS:

Research Approach:

The quantitative research approach was used to assess the perception of evaluators while evaluating university theory answer sheets.

Research Design:

A Descriptive research design was chosen for the study to assess the perception of evaluators while evaluating university theory answer sheets.

Study Setting:

The study was conducted in north-east and north-west sates of India.

Population:

Target population: The target population were all the evaluators who evaluate the university theory answers sheets.

Accessible population:

The accessible population was the evaluators who voluntarily participated in the study.

Sample Size:

The sample includes 114 evaluators from nursing profession.

Sampling Technique:

Purposive sampling, a type of non-probability sampling technique, was employed for the data gathering.

Sampling Criteria:

Inclusion Criteria:

The study includes evaluators who were;

· Currently working.

· Present during the time of data collection.

· Willing to participate in the study.

Exclusion Criteria:

The study excludes the evaluators who were not;

· Retired professionals.

· Evaluators of diploma courses in nursing.

DEVELOPMENT OF TOOL

Tool used for present study is developed specifically as per the need of the study. After the intense library search tool of the data was divided into two parts.

PART I: This part includes socio demographic variables i.e. Age (in years), Gender, working area, qualification and year of experience.

PART II: Self structured perception questionnaire.

Scoring Procedure:

There were 12 items to assess the perception of evaluators while evaluating theory answer sheets. Criterion measure used in the study was as follows:

Perception	Scores
Negative	0-5
Positive	6-12

CONTENT VALIDITY:

The prepared tool along with the objectives were submitted to experts in the field of community health nursing, obstetrical nursing, medical surgical nursing, child health nursing, psychiatric department. Experts were requested to judge the item for clarity, relatedness, meaningfulness and content. Suggestion from all experts was received and changes were according to them. Finally, the tool's components were reorganised, and validated tools were prepared to collect data from sample individuals.

RELIABILITY OF TOOL:

The split-half approach was used to determine the tool's dependability. The statistical analysis spearman’s rank co-relation coefficient was found to be ‘r’ value was 0.74 which was positively correlated. Thus, the instrument's dependability was determined.

Data Collection Procedure:

Data collection was done through virtual mode (Google form), which allows its distribution as a link to the website where it is hosted and therefore its disseminated via the internet and possibility to reply to it from the electronic devices or from computer with internet access. Consent was obtained from the participant based on the mode of conduct and assures them that to confidentiality will be maintained. All the subjects were very much cooperative and the investigators expressed their gratitude for their co-operation.

ETHICAL CONSIDERATIONS:

· Written informed consent will be taken from each study sample.

· Confidentiality of the study sample will be maintained.

PLAN FOR DATA ANALYSIS:

By using descriptive and inferential statistics such as frequency, mean percentage, and chi-square test, the data was examined in light of the study's objectives.

Table 1.1: Frequency And Percentage Distribution of Subject as Per Their Socio Demographic Variables. N=114

Socio Demographic Variables	Frequency	Percentage
1. Age (In Years)
a) Below 30	29	25.4 %
b) 31-40	64	56.1 %
c) 41-50	18	15.8 %
d) Above 51	3	2.6 %
2. GENDER
(a) Male	20	17.5 %
(b) Female	94	82.5 %
(c) Others	0	0 %
3. WORKING AREA
(a) Govt. sector	7	6.1 %
(b) Private sector	107	93.9%
4.QUALIFICATION
a) M.Sc. Nursing	90	78.9%
b) Ph.D. Nursing	15	13.2%
c) Additional to M.Sc. Nursing	9	7.9 %
5. YEAR OF EXPERIENCE
(a) 3-7 Years	44	38.6 %
(b) 8-12 Years	37	32.5 %
(c) 13-16 Years	18	15.8 %
(d) Above 16	15	13.2 %

Table: 1.2 Overall Perception and Category Wise Responses of Evaluators While Evaluating The University Theory Answers Sheets. N=114

S. No.	Self-Structured Questionnarie	PERCEPTION				Mean	S.D
		NEGATIVE		POSITIVE
		N	%	N	%
1.	Do you think that adherence to the university guidelines while checking theory answer sheets is important to follow?	4	3.5%	110	96.5%	0.96	0.185
2.	Do you think that answers written in chronological order helps to assess university theory answers appropriately?	12	10.5%	102	89.5%	0.89	0.308
3.	Do you think that writing of the students causes hindrance in the proper assessment of university theory answers?	33	28.9%	81	71.1%	0.71	0.456
4.	Do you think that family stress or workplace stress effect the evaluation of university theory answer sheets?	57	50%	57	50%	0.50	0.502
5.	Do you think that length of the answer or depth of analysis matter the evaluation of university theory answer sheets?	28	24.6	86	75.4%	0.75	0.432
6.	Do you think that good presentation of answer (Highlighted important point/well labeled diagrams) sheets help the evaluator to assess the theory sheets appropriately?	2	1.8%	112	98.2%	0.98	0.132
7.	Do you think that answers written under headings and subheadings helps in easy evaluation of university theory answers sheets?	4	3.5%	110	96.5%	0.96	0.185
8.	Do you think that number of university theory answer sheets to be evaluated affects the perception of evaluator?	51	44.7%	63	55.3%	0.55	0.499
9.	Do you think that time limit given for the submission of result effects the evaluation process of the evaluator?	41	36.0%	73	64.0%	0.64	0.482
10.	Do you think that knowledge related to specialty impact the evaluation of university theory answer sheets?	9	7.9%	105	92.1%	0.92	0.271
11.	Do you think that bad or good mood if is an impediment while assessing the university theory answer sheets?	59	51.8%	55	48.2%	0.48	0.502
12.	Do you think that checking previous theory answer sheet will affect the marking of the next answer sheet?	68	59.6%	46	40.4%	0.40	0.493

Table 1.3: Association Between Levels of Perception of Evaluators While Evaluating University Theory Answers Sheets with Their Selected Socio- Demographic Variables. N=114

Socio-Demographic Variables	Perception		df	Chi-square
	Positive	Negative
1. Age (In Years)			3	0.090^NS
a) Below 30	29	0
b) 31-40	58	6
c) 41-50	15	3
d) Above 51	2	1
2. Gender			1	0.831^NS
a) Male	18	2
b) Female	86	8
c) Others	0	0
3. Working area			1	0.595^NS
a) Govt. sector	6	1
b) Private sector	98	9
4. Qualification			2	0.755^NS
a) M.Sc. Nursing	83	7
b) Ph.D. Nursing	13	2
c) Additional to M.Sc. (N)	8	1
5. Year of experience			3	0.054^NS
(a) 3-7 Years	42	2
(b) 8-12 Years	35	2
(c) 13-16 Years	16	2
(d). Above 16	11	4

*NS= Non-Significant

MAJOR FINDING:

· Findings of the study revealed that maximum 56.1% of evaluators were belongs to the age group of 31-40 years followed by 82.5% were females.

· Distribution based on the working area revealed that majority 93.9 % were working in private area followed by 78.9 % were qualified M.Sc (N).

· Out of 114 evaluators majority 38.6% of evaluators were having 3 to7 years of experience.

· Distribution based on the perception of evaluators revealed that majority 96.5% of evaluators follow university guidelines while evaluating the university theory answers sheets.

· 89.5% of evaluators perceive that answers written in chronological orders helps to assess the university theory answers sheets appropriately.

· 71.1% of evaluators perceive that writing of the students causes hindrance in the proper assessment of university theory answers.

· 50% of evaluators have the positive perception that workplace stress or family stress effect the evaluation of university theory answers sheets.

· 75.4% of evaluators has positive perception that length of the answers or depth of analysis matter while evaluating the university theory sheets.

· Majority 98.2% of evaluators has positive perception about the presentation of answers while evaluating the university theory answers sheets.

· Majority 96.5% of evaluators believed that answers written under headings and subheadings helps in easy evaluation.

· 55.3% of evaluators perceive that number of theory answers sheets to be evaluated affect the perception of evaluators.

· 64% of evaluators has positive perception regarding the time limit given for the submission of result.

· Majority 92.1% of evaluators has positive perception that knowledge related to speciality impact the evaluation.

· Majority 51.8% of evaluators has negative perception regarding bad or good mood impediment while assessing the university theory answer sheets.

· Majority 59.6% of evaluators has negative perception regarding checking previous theory sheet will affect the marking of the next answer sheets. 

· The present study confirmed that the variables like age, gender, working area, qualification and years of experience were found to be statistically non-significant i.e., p<0.05.

DISCUSSION:

The findings of present study were supported by similar study conducted by GRK Sarma, Saji K. John, Thomas Mathew, Venkatesh Aiyagari, Sivaraman Nair, Gareth J. Parry, Satish Khadilkar, and Parthasarathy Satishchandra (2022) on Neurology Exit Examination System In India: A Survey Of Examiners’ Perceptions And Recommendations. The study was conducted as online survey of examiners perceptions and recommendations using a set of 10 multiple-choice questions and an open-ended question. Results revealed that 46 examiners provided completed responses suitable for analysis. Nearly equal proportions (30%) of the examiners had 10 years, 10–25 years and >25 years’ experience. 92% were not satisfied with current system, 95% did not find adequate time for correction of theory scripts, 90% felt that theory questions were random, and 95% had legibility issues. 84% felt that the practical exams do not test true learning, 98% felt the examination stress impairs the performance and 85% felt that there are no objective criteria to pass the candidate. 83% felt the current system-needed changes. The key suggestions provided by the examiners to improve the system included objective assessments like MCQ, OSCE, OSLER and DOPS, inclusion of larger number of short answer type questions and periodic internal assessments of the candidates⁹.

CONCLUSION:

The study's findings concluded that evaluators are often perceived as impartial and objective assessors who base their judgments on evidence and data rather than personal biases or preferences. Their ability to provide unbiased evaluations is typically highly valued. Evaluators are often seen as critical thinkers who can analyze complex information, identify strengths and weaknesses, and provide constructive feedback to the students. In some context, they are held responsible for their actions, decisions, and outcomes. Overall, the perception of evaluators is shaped by factors such as their professionalism, expertise, integrity, and the perceived fairness and utility of their evaluations¹⁰.

FUNDING AND DISCLOSURES:

The study was conducted without funding. There is no potential conflict of interest.

REFERENCES:

1. https://www.britannica.com/topic/perception

2. https://www.eklavvya.com/blog/challenges-answer-sheet-evaluation/

3. https://superteacher.co/blog/5-effective-ways-to-reduce-time-during-answer-sheet-evaluation-process

4. https://files.eric.ed.gov/fulltext/ED561139.pdf

5. https://pubmed.ncbi.nlm.nih.gov/34924781/

6. https://journals.pan.pl/Content/118020/PDF/2020-02-FOMC-09-Stalmach-Przygoda-etal.pdf

7. https://www.researchgate.net/publication/43347428_Examiner_perceptions_of_a_portfolio_assessment_process

8. https://www.researchgate.net/publication/238399849_Exploring_the_Nature_of_Examiner_Thinking_during_the_Process_of_Examination_Marking/citation/download

9. Sarma G, John SK, Mathew T, Aiyagari V, Nair S, Parry GJ, Khadilkar S, Satishchandra P. Neurology Exit Examination System in India: A Survey of Examiners' Perceptions and Recommendations. Ann Indian Acad Neurol. 2022; 25(2): 189-193. doi: 10.4103/aian.aian_666_21. Epub 2022; Apr 6. PMID: 35693660; PMCID: PMC9175401.

10. Majumder MAA, Kumar A, Krishnamurthy K, Ojeh N, Adams OP, Sa B. An evaluative study of objective structured clinical examination (OSCE): students and examiners perspectives. Adv Med Educ Pract. 2019; 10: 387-397. doi: 10.2147/AMEP.S197275. PMID: 31239801; PMCID: PMC6556562.

11. A. K. Srinivas. The Evaluation of Self Help Groups- Bank Linkage Programme. Asian J. Management. 2010; 1(1): 30-32

12. Shailaja P. Yadav. Performance evaluation of public sector banks with reference to ‘CAMEL model’ for the period of 2006 – 2013. Asian J. Management. 2014; 5(4): 374-379

13. Mehrnoosh Pazargadi, Tahereh Ashktorab, Sharareh Khosravi. Nursing Students’ Experiences on the Evaluating Role of Their Clinical Educators: a Qualitative Study. Asian J. Nur. Edu. and Research. 2012; 2(3): 149-153.

14. Mamata Prasad. An evaluation of the quality of work life: A study of the public sector nurses in West Bengal. Asian J. Nur. Edu. and Research. 2016; 6(2): 199-203.

15. Smita Sunil Gawade. Students Evaluation of an Observed Structured Clinical Evaluation (OSCE) in Community Health Nursing Clinical Practice. Asian J. Nursing Education and Research. 2018; 8(2): 278-281.

16. Hanan Ramzy Ahmed Atalla, Gehan. H. Soliman, Warda Mohamed Henedy. Evaluation of Conservative Nursing Measures among Patients with Lower Limbs Varicose Vein. Asian J. Nursing Education and Research. 2019; 9(3): 449-459

17. Betcy George, Sheela Shenai N. A. Curriculum Evaluation - A Cross-Sectional Study under selected Nursing College of KUHS. Asian Journal of Nursing Education and Research. 2021; 11(4): 485-7.

18. Durai Anand Kumar, V. Balakrishnan. Corporate Social Responsibility: Evaluation of Existing Practices at UAE. Asian J. Management. 2011; 2(3): 108-114.

19. Audumbar Digambar Mali, Ritesh Bathe. An Updated Review on Formulation and Evaluation of Microsponge. Research J. Topical and Cosmetic Sci. 2015; 6(2): 77-85

Received on 09.07.2024         Revised on 18.09.2024

Accepted on 27.10.2024      Published on 16.12.2024

Available online on December 31, 2024

Int. J. Nursing Education and Research. 2024;12(4):225-230.

DOI: 10.52711/2454-2660.2024.00048

©A and V Publications All right reserved